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Abstract 

Shannnon entropy is an efficient tool to measure uncertain information. How- 
ever, it cannot handle the more uncertain situation when the uncertainty is 
represented by basic probability assignment (BPA), instead of probability 
distribution, under the framework of Dempster Shafer evidence theory. To 
address this issue, a new entropy, named as Deng entropy, is proposed. The 
proposed Deng entropy is the generalization of Shannnon entropy. If un- 
certain information is represented by probability distribution, the uncertain 
degree measured by Deng entropy is the same as that of Shannnon’s en- 
tropy. Some numerical examples are illustrated to shown the efficiency of 
Deng entropy. 
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1. Introduction 



Uncertainty is ubiquitous in nature. How to measure the uncertainty has 
attracted much attention [1, 2], Various categorizations exist to accommo- 
date different kinds of uncertainties. Numerous uncertainty theories has been 
developed, such as probability theory [3], fuzzy set theory [4], possibility the- 
ory [5], Dempster-Shafer evidence theory [6, 7], rough sets[8], DSmT[9, 10], 
generalized evidence theory [11] and D numbers [12, 13, 14, 15]. 

Since firstly proposed by Clausius in 1865 for thermodynamics [16], the 
study of uncertainty and entropy attracts great interests and various types 
of entropies are developed, such as information entropy [17], Tsallis entropy 
[18], nonadditive entropy [19, 20, 21], Information entropy [17], derived from 
the Boltzmann-Gibbs (BG) entropy [22] in thermodynamics and statistical 
mechanics, has been an indicator to measures aleatoric uncertainty which is 
associated with the probability density function (PDF). 

For the aleatoric uncertain information expressed by PDF, information 
entropy proposed by Shannon [17] is a good measure. However, with respec- 
t to other uncertain information including epistemic, irreducible, reducible 
and inferential uncertainty, classical information entropy is invalid. In this 
paper, one of these uncertainties, epistemic uncertainty, is taken into consid- 
eration. Dempster-Shafer theory evidence theory[6, 7] is mainly proposed to 
handle such uncertainty. In Dempster-Shafer evidence theory, the epistemic 
uncertainty simultaneously contains nonspecificity and discord [23] which 
are coexisting in a basic probability assignment function (BPA). Several un- 
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certainty measures, such as AU [24, 25], AM [23], have been proposed to 
quantify such uncertainty in Dempster-Shafer theory. What’s more, five ax- 
iomatic requirements have been further built in order to develop a justifiable 
measure. These five axiomatic requirements are range, probabilistic consis- 
tency, set consistency, additivity, subadditivity, respectively [26]. However, 
existing methods are not efficient to measure uncertain degree of BPA. To 
address this issue, a new entropy, named as Deng entropy, is proposed in this 
paper. 

The paper is organized as follows. The preliminaries Dempster-Shafer 
evidence theory and entropy are briefly introduced in Section 2. Section 3 
presents Deng entropy. Some numerical examples are illustrated in Section 
4 to show the efficiency of Deng entropy. Finally, this paper is concluded in 
Section 5. 

2. Preliminaries 

In this section, some preliminaries are briefly introduced. 

2.1. Dempster-Shafer evidence theory 

Dempster-Shafer theory (short for D-S theory), also called belief function 
theory, as introduced by Dempster [6] and then developed by Shafer [7], has 
emerged from their works on statistical inference and uncertain reasoning. 
This theory is widely applied to uncertainty modeling [27, 28], decision mak- 
ing [29, 30, 31, 32, 33, 34, 35, 36, 37]and information fusion [38, 39] and 
uncertain information processing [40]. D-S theory mainly focus on the epis- 
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temic uncertainty, but it is also valid for aleatoric uncertainty. It has many 
merits by contrast probability theory. First, D-S theory can handle more 
uncertainty in real world. In contrast to the probability theory in which 
probability masses can be only assigned to singleton subsets, in D-S theory 
the belief can be assigned to both singletons and compound sets. Second, in 
D-S theory, prior distribution is not needed before the combination of infor- 
mation from individual information sources. Third, D-S theory allows one to 
specify a degree of ignorance in some situations instead of being forced to be 
assigned for probabilities. Some basic concepts in D-S theory are introduced. 

Let X be a set of mutually exclusive and collectively exhaustive events, 
indicated by 



where set X is called a frame of discernment. The power set of X is indicated 
by 2 X . namely 



For a frame of discernment X = { 61 , 62 , ■ ■ ■ , 6 \x\}, a mass function is a 
mapping m from 2 X to [0, 1], formally defined by: 






( 1 ) 




m : 



2 X -> [ 0 , 1 ] 



( 3 ) 



which satisfies the following condition: 



m(0) = 0 and 




( 4 ) 



Ae2 x 
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In D-S theory, a mass function is also called a basic probability assignment 
(BPA). Assume there are two BPAs indicated by mi and m 2 , the Dempster’s 



rule of combination is used to combine them as follows: 



m(A) 




( 5 ) 



with 



K = ^2 rni(B)m 2 (C) 



( 6 ) 



unc=0 



Note that the Dempster’s rule of combination is only applicable to such two 
BPAs which satisfy the condition K < 1. 

D-S theory has more advantages in in handling uncertainty compared 
with classical probability theory. When information is adequate, probability 
theory is effective to handle that situation. However, when information is 
not adequate, probability theory is invalid to such uncertain situation. Here 
is an example. 

As shown in Figure 1, assume there are two boxes. There are red balls the 
left box, and green balls in the right box. The number of balls in each box 
is unknown. Now, a person is assigned to pick a boll from these two boxes. 
We know that he chooses the the left box with a probability PI = 0.6, and 
chooses the right box with a probability P 2 = 0.4. Based on probability 
theory, it can be obtained that the probability of picking a red ball is 0.6, 
the probability of picking a green ball is 0.4, namely p(R) = 0.6, p(G) = 0.4. 

Now, let us change the configuration, as shown in Figure 2. In the left box, 
there are still only red balls. But in the right box, there are not only red balls 
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PI =0.6 P2=0.4 



Figure 1: A game of picking ball which can be handled by probability theory 

but also green balls. In accord with above, the exact number of balls in each 
box is still unknown, and the ratio of them are completely unknown. This 
person also has 0.6 probability to choose the left box and 0.4 probability 
to choose the right box. The question is how possible that a red ball is 
picked. Obviously, in this case due to lack of adequate information, p(R) 
and p(G) cannot be obtained. Facing the situation of inadequate information, 
probability theory is incapable. However, if using D-S theory to analyze this 
problem, we can obtain a BPA that m(R) = 0.6 and m(R , G) = 0.4, which 
means the probability of red ball being picked is 0.6 and the probability of 
red ball or green ball being picked is 0.4. In the framework of D-S theory, the 
uncertainty has been expressed more effective. D-S theory has more ability 
to express uncertain information than probability theory. 



6 




PI =0.6 P2=0.4 



Figure 2: A game of picking ball where probability theory is unable but D-S theory is able 
to handle 

2.2. Existing entropy and open issue 

Entropy is associated with uncertainty, and it has been a measure of 
uncertainty and disorder. The concept of entropy is derived from physics 
[16]. In thermodynamics and statistical mechanics, the entropy often refers 
to Boltzmann-Gibbs entropy [22]. According to Boltzmann’s H theorem, the 
Boltzmann-Gibbs (BG) entropy of an isolated system Sbg is obtained in 
terms of the probabilities associated the distinct microscopic states available 
to the system given the macroscopic constraints, which has the following 
form 

w 

Sbg = -k^pilnpi (7) 

1=1 

where k is the Boltzmann constant, W is the amount of distinct microscopic 

states available to the isolated system, pi is the probability of microscopic 
w 

state i satisfying = 1- Equal probabilities, i.e. Wi,Pi = 1/IE, is a 

i=l 
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particular situation. In that situation, BG entropy has the following form 



S BG = k In W 



( 8 ) 



In information theory, Shannon entropy [17] is often used to measure the 
information volume of a system or a process, and quantify the expected value 
of the information contained in a message. Information entropy, denoted as 
H, has a similar form with BS entropy 

N 

H = -^PilogbPi (9) 

i = 1 

where N is the amount of basic states in a state space, p t is the probability of 

w 

state i appears satisfying 'Yh'Pi — 1, b is base of logarithm. When b = 2, the 

i= 1 

unit of information entropy is bit. If each state equally appears, the quantity 
of H has this form 

H = log 2 N (10) 

In information theory, quantities of H play a central role as measures of 
information, choice and uncertainty. For example, the Shannon entropy of 
the game shown in Figure 1 is H — 0.6 x log 2 0.6 + 0.4 x log 2 0.4 = 0.9710. 

But, it is worthy to notice that the uncertainty of this game shown in Figure 

2 cannot be calculated by using the Shannon entropy. 

According to mentioned above, no matter the BG entropy or the informa- 
tion entropy, the quantity of entropy is always associated with the amount 
of states in a system. Especially, for the case of equal probabilities, the en- 



tropy or the uncertainty of a system is a function of the quantity of states. 
Moreover, in that particular case, the entropy is the maximum. 

3. Deng entropy 

With the range of uncertainty mentioned above, Deng entropy can be 
presented as follows 

E d = -J2 m ^ l ° g 2 i%l ( n ) 

i 

where, F) is a proposition in mass function m, and \Ff is the cardinality 
of Fi. As shown in the above definition, Deng entropy, formally, is similar 
with the classical Shannon entropy, but the belief for each proposition F* is 
divided by a term (2^ — 1) which represents the potential number of states 
in Fi (of course, the empty set is not included). 

Specially, Deng entropy can definitely degenerate to the Shannon entropy 
if the belief is only assigned to single elements. Namely, 

E d = - m{0i) log = 

i i 

4. Numerical examples and discussions 

In the section, a lot of examples are given to show the effectiveness of 
Deng entropy. 

Example 1. Assume there is a mass function m(a) = 1, the associated 
Shannon entropy FI and Deng entropy Ed are calculated as follows. 

H — lx log 1 = 0 
E d = lx log j- = 0 
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Example 2. Given a frame of discernment X = {a, b , c}, for a mass func- 
tion m(a) = m(b ) = m(c) = 1/3, the associated Shannon entropy H and 
Deng entropy Ed are 



H = x log i i x log i i x log i = 1.5850 



E d = 



1 1/3 

- X log —r 

3 & 2 1 - 1 



1 1/3 

X log —r 

3 6 2 1 — 1 



1 1/3 

X log —r 

3 S 2 1 - 1 



= 1.5850 



Clearly, Example 1 and 2 have shown that the results of Shannon entropy 
and Deng entropy are identical when the belief is only assigned on single 
elements. 

Example 3. Given a frame of discernment X = (a, b , c}, for a mass func- 
tion m(a , 6 , c) = 1 , 



E d = 



— 1 x log 



1 

2^1 



2.8074 



For another mass function m(a ) = m(b ) = m(c) = m(a,b) = m(a,c) = 
m(b, c ) = m(a, b, c ) = 1/7, 



Ed = -i X log - \ x log - i X log 

-i X log - i x log - y X log ~^T~\ - J X log 
= 3.8877 



1/7 

23-1 



Example 4. Given a frame of discernment X = {cp, a 2 , ■ ■ ■ , cln}, let us 
consider three special cases of mass functions as follows. 

• mi(Fi) = mi(Fj) and (Ff) = 1, \/F i: Fj C X, F i: Fj ± 0. 

i 

• m 2 (X) = 1. 

• m 3 (ai) = m 3 (a 2 ) = • ■ ■ = m 3 (a N ) = 1/N. 

Their associated Deng entropies change with N , as shown in Figure 3. 
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Figure 3: Deng entropy as a function of the size of frame of discernment in three types of 
mass functions 

Example 5. Given a frame of discernment X with 15 elements which are 
denoted as element 1, element 2, etc. A mass function is shown as follows. 

m({3, 4, 5}) = 0.05, m({6}) = 0.05, m(A) = 0.8, m(X) = 0.1 

Table 1 lists various Deng entropies when A changes, which is graphi- 
cally shown in Figure f. The results shows that the entropy of m increases 
monotonously with the rise of the size of subset A. It is rational that the 
entropy increases when the uncertainty involving a mass function increases. 
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Table 1: Deng entropy when A changes 



A = { 1, - - - , 4} 4.0118 

7l = {1, ■ ■ ■ , 5} 4.5925 

A = {1, ■ ■ ■ , 6} 5.1599 

A = { 1, - - - , 7} 5.7207 

A = {1, • - - , 8} 6.2784 

A = {1, • - - , 9> 6.8345 

A = {1, - - - , 10} 7.3898 

A = { 1, - - - , 11} 7.9447 

A = { 1, - - - , 12} 8.4994 

A = {1, - - - , 13} 9.0540 

A = {!,•••, 14} 9.6086 



Cases 



Deng entropy 



71 = { 1 } 

71 = { 1 , 2 } 

71 = {1,2, 3} 



1.8454 

2.7242 

3.4021 
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Deng entropy 




Figure 4: Deng entropy as a function of the size of A 
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5. Conclusion 



Dempster Shafer evidence theory has widely used in many application- 
s due to its advantages to handle the aleatoric and epistemic uncertainty. 
However, how to measure uncertain degree in evidence theory is still an open 
issue. The main contribution of this paper is that a new entropy, named 
as Deng entropy, is presented. Deng entropy is the generalization of Shan- 
non entropy. In the case when the BPA is degenerated as probability, Deng 
entropy is degenerated as Shannon entropy. However, Numerical examples 
are illustrated to show the efficiency of Deng entropy. Some properties of 
Deng entropy are discussed. The new entropy provides a promising way to 
measure uncertain degree. One of the ongoing works is to present the cross 
entropy of BPA, which can be easily derived. 
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